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DETAILED ACTION 
Response to Amendment 

1 . This office action is in response to the amendment filed on 05 February 2007. 

Response to Arguments 

2. Applicant's arguments with respect to claims 1 , 9, and 1 1 have been considered 
but are moot in view of the new ground(s) of rejection. 

Drawings 

3. The drawings filed on 23 April 2002 are accepted by the Examiner. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

5. Claims 1-6, 9-14, 16, and 18-20 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Colmenarez et al. (US 6,931,147). 

6. Regarding Claim 1 , Colmenarez et al disclose a method for collecting and 
associating affective information for a plurality of images in an imaging system, 
comprising the steps of: a) displaying a plurality of digital images for viewing by a 
particular user (Fig. 1 A, step 145) (col. 2, lines 25-36); b) automatically collecting 
affective information for the plurality of digital images as the particular user views the 
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images (Fig. 1B, step 155) (col. 2, lines 42-48); c) storing in a database the collected 
affective information for each of the plurality of digital images and associating the 
collected affective information with the particular user (Fig. 1 B, step 165) (col. 2, lines 
49-51 ); and d) using the stored collected affective information associated with the 
particular user to facilitate retrieval of particular digital images from the plurality of digital 
images viewed in step a (Fig. 1A, step 125) (col. 2, lines 12-25). 

7. Regarding Claim 2, Colmenarez et al disclose a method further including the step 
of: e) the particular user providing a personal identifier (website requests an identifier 
and password) (col. 2, lines 1-11). 

8. Regarding Claim 3, Colmenarez et al disclose a method wherein the affective 
information and a user identifier are stored with each of the digital images in a plurality 
of digital image files (identifying information and correlation module) (col. 5, lines 1-15). 

9. Regarding Claim 4, Colmenarez et al disclose a method wherein each of the 
digital image files includes affective information and user identifiers for a plurality of 
users (categories can be personalized) (col. 5, lines 27-38). 

10. Regarding Claim 5, Colmenarez et al disclose a method wherein the step of 
automatically collecting affective information includes monitoring the physiology of the 
particular user (mood of a viewer) (col. 3, lines 18-35). 

1 1 . Regarding Claim 6, Colmenarez et al disclose a method wherein the step of 
automatically collecting affective information uses a video camera (camera 206) (col. 2, 
lines 54-65). 



I 
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12. Regarding Claim 9, Colmenarez et al disclose a method for providing affective 
information for images captured by a particular user in an imaging system, comprising 
the steps of: a) sequentially displaying a plurality of digital images captured by the 
particular user for viewing by a particular user (Fig. 1 A, step 145) (col. 2, lines 25-36); b) 
automatically collecting affective information related to the particular user's reaction to 
each of the plurality of digital images (Fig. 1B, step 155) (col. 2, lines 42-48); c) storing 
the collected affective information for each of the plurality of digital images and 
associating the collected affective information with the particular user (Fig. 1 B, step 165) 
(col. 2, lines 49-51), and d) using the stored collected affective information associated 
with the particular user to facilitate retrieval of particular digital images from the plurality 
of digital images viewed in step a (Fig. 1 A, step 125) (col. 2, lines 12-25). 

13. Regarding Claim 10, Colmenarez et al disclose a method wherein the collected 
affective information for each of the plurality of digital images is stored along with the 
digital image in separate digital image files, and the digital image files include a user 
identifier which identifies the particular user (personalized categories) (col. 5, lines 27- 
38). 

14. Regarding Claim 1 1 , Colmenarez et al disclose a system for providing affective 
information for images captured by a particular user in an imaging system, comprising: 
a) a digital memory which stores a set of the digital images captured by the particular 
user (database 225) (col. 4, lines 5-13); b) means for identifying the particular user 
(website requests an identifier and password) (col. 2, lines 1-1 1); c) a display which 
sequentially displays the set of stored digital images for viewing by the particular user 
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(photos displayed as a virtual mood based photo album) (col. 2, lines 25-36); d) a 
sensor for automatically measuring the particular user's reaction to the image as the 
particular user views the image (second image of the facial expression is captured by 
the camera) (col. 2, lines 42-48); e) a processor for processing the signal from the 
sensor to provide affective information associated with each image in the set of digital 
images (image of the facial expression of the viewer is provided to a pattern recognition 
module of a processor) (col. 2, lines 2-1 1 ); f) a memory for storing the affective 
information associated with each image in the set of digital images (set of photos 
retrieved from a storage area, the photos corresponding to the type of mood detected) 
(col. 2, lines 18-24); and g) using the stored collected affective information associated 
with the user for causing the processor to retrieve particular digital images from the set 
of digital images (set of photos retrieved from a storage area, the photos corresponding 
to the type of mood detected) (col. 2, lines 18-24). 

15. Regarding Claim 12, Colmenarez et al disclose a system wherein the sensor is a 
video camera (camera 206) (col. 2, lines 54-65). 

16. Regarding Claim 13, Colmenarez et al disclose a system wherein the processor 
processes the signal from the video camera in order to determine the particular user's 
facial expression (second image of the facial expression is captured by the camera) 
(col. 2, lines 42-48). 

17. Regarding Claim 14, Colmenarez et al disclose a system wherein the sensor 
measures the particular user's biometric response (monitoring of facial expressions) 
(col. 2, lines 42-48). 
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18. Regarding Claim 16, Colmenarez et al disclose a system wherein the system 
includes a pointing device, and the sensor is incorporated into the pointing device 
(keyboard, joystick or mouse) (col. 2, lines 25-36). 

19. Regarding Claim 18, Colmenarez et al disclose a system wherein the affective 
information is stored in the digital memory (database 225) (col. 4, lines 5-13). 

20. Regarding Claim 19, Colmenarez et al disclose a system wherein the affective 
information is stored with each digital image in a digital image file (photos characterized 
at least by mood) (col. 4, lines 5-13). 

21 . Regarding Claim 20, Colmenarez et al disclose a system wherein the digital 
image file includes affective information and user identifiers for a plurality of users 
(personalized categories) (col. 5, lines 27-38). 

Claim Rejections - 35 USC § 103 

22. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

23. Claims 7 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Colmenarez et al. in view of Philomin et al. (US 2003/0081834). 

24. Regarding Claim 7, Colmenarez et al fail to teach a method wherein the step of 
automatically collecting affective information includes determining the duration of time 
the particular user views each of the plurality of images. 
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25. Philomin et al teach a method wherein the step of automatically collecting 
affective information includes determining the duration of time the particular user views 
each of the plurality of images (facial expression of viewer for a predetermined amount 
of time) (page 4, Claim 4). 

26. Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have combined the teaching of Colmenarez with the teaching of 
Philomin to detect the emotional state of the viewer based on how long the viewer views 
the images. 

27. Regarding Claim 8, Colmenarez et al fail to teach a method wherein the step of 
automatically collecting affective information for the plurality of digital images includes 
monitoring the gaze of the particular user. 

28. Philomin et al teach a method wherein the step of automatically collecting 
affective information for the plurality of digital images includes monitoring the gaze of 
the particular user (contour of the eye) (page 3, paragraph [0028]). 

29. Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have combined the teaching of Colmenarez with the teaching of 
Philomin to detect the emotional state of the viewer based the gaze of the viewer. 

30. Claims 5, and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Colmenarez et al. in view of Strubbe et al. (US 6,795,808). 

31 . Regarding Claim 1 5, Colmenarez et al fail to teach a system wherein the sensor 
measures the particular user's galvanic skin response. 
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32. Strubbe et al teach a system wherein the sensor measures the particular user's 
galvanic skin response (galvanic skin response sensor) (col. 21, lines 47-67, col. 22, 
lines 1-6). 

33. Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have combined the teaching of Colmenarez with the teaching of 
Strubbe to use a biometric sensor to monitor biological or physiological characteristics 
or responses of the viewer. 

34. Regarding Claim 17, Colmenarez et al fail to teach a system wherein the sensor 
measures the particular user's galvanic skin response. 

35. Strubbe et al teach a system wherein the sensor measures the particular user's 
galvanic skin response (galvanic skin response sensor) (col. 21 , lines 47-67, col. 22, 
lines 1-6). 

36. Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to have combined the teaching of Colmenarez with the teaching of 
Strubbe to use a biometric sensor to monitor biological or physiological characteristics 
or responses of the viewer. 

Conclusion 

37. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 



i 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Satwant K. Singh whose telephone number is (571) 
272-7468. The examiner can normally be reached on Monday thru Friday 8am - 
4:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David K. Moore can be reached on (571) 272-7437. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 



Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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